fix(sc): keep the watchdog armed while the train pump drains - #3783

run() waits once with FIRST_COMPLETED, handles whichever task finished, then
falls through to a bare `await train_task`. From that point the watchdog and
the fleet probe are still running but nothing awaits them again, and the
`finally` retrieves their exceptions with gather(return_exceptions=True) and
discards them -- so not even asyncio's "Task exception was never retrieved"
warning fires.

The window this opens is the normal path, not an edge case. The rollout pump
finishing first is end-of-data, as run()'s own comment says: "A normally
exhausted rollout pump leaves the train pump to drain committed groups." For
the whole of that drain, a RolloutStall under stall_action: abort, a
GenerationFleetExhausted, or an env-health abort lands on an unobserved task
while run() parks on the wedged train pump. The job then holds its GPUs until
the scheduler's wall clock kills it, with nothing in the driver log -- which is
what WatchdogConfig, documented as "Last-resort detection for stalls that no
other layer catches", exists to prevent.

Loop the wait instead, so every task stays supervised until the train pump
exits. The priority order (probe, watchdog, rollout, train) is unchanged, and
so is the behaviour when the train pump finishes first.

Three tests. The drain-phase one fails on main with a TimeoutError rather than
the RolloutStall it asserts; the other two pin the paths that already worked.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: Tianyi Zhang <123608656+tianyi-zhang-02@users.noreply.github.com>